Method and system for using external storage to amortize CPU cycle utilization

ABSTRACT

A method and system for using external storage to amortize CPU cycle utilization, wherein translated instructions are stored in a storage medium and subsequently accessed on a subsequent execution of a non-native application in order to amortize CPU cycles used in generating the translated instructions.

This application is a Continuation of U.S. patent application Ser. No.10/335,405, filed Dec. 30, 2002, entitled “A METHOD AND SYSTEM FOR USINGEXTERNAL STORAGE TO AMORTIZE CPU CYCLE UTILIZATION” to O'Clair, et al.,which is hereby incorporated herein in its entirety.

TECHNICAL FIELD

The present invention relates generally to digital computer systems.More specifically, the present invention pertains to reducing executionoverhead imposed on the processing environment of a digital computersystem.

BACKGROUND ART

Many types of digital computer systems utilize translation or emulationto implement software based functionality. Generally, translation andemulation both involve examining a program of software instructions andperforming the functions and actions dictated by the softwareinstructions, even though the instructions are not “native” to thecomputer system. In the case of translation, the non-native instructionsare converted, or translated, into a form of native instructions whichare designed to execute on the hardware of the computer system. Examplesinclude prior art translation software and/or hardware that operateswith industry standard x86 applications to enable the applications toexecute on non-x86 or alternative computer architectures. Generally, atranslation process utilizes a large number of processor cycles, andthus, imposes a substantial amount of overhead. The performance penaltyimposed by the overhead can substantially erode any benefits provided bythe translation process. Thus what is required is a solution that canimplement a computer software translation process while reducing theoverhead penalty imposed on a user.

DISCLOSURE OF THE INVENTION

Embodiments of the present invention provide a method and system forusing external storage to amortize CPU cycle utilization.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and form a part ofthis specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention:

FIG. 1 shows a first diagram of the operation of a system used toimplement an instruction translation process in accordance with oneembodiment of the present invention.

FIG. 2 shows a second diagram of the basic components of a system usedto implement an instruction translation process in accordance with oneembodiment of the present invention.

FIG. 3 shows a third diagram of the basic components of a system used toimplement an instruction translation process in accordance with oneembodiment of the present invention.

FIG. 4 shows a flowchart of the steps of a CPU cycle amortizationprocess in accordance with one embodiment of the present invention.

FIG. 5 shows the general components of a computer system in accordancewith one embodiment of the present invention.

BEST MODES FOR CARRYING OUT THE INVENTION

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings. While the invention will be described in conjunction with thepreferred embodiments, it will be understood that they are not intendedto limit the invention to these embodiments. On the contrary, theinvention is intended to cover alternatives, modifications andequivalents, which may be included within the spirit and scope of theinvention as defined by the appended claims. Furthermore, in thefollowing detailed description of embodiments of the present invention,numerous specific details are set forth in order to provide a thoroughunderstanding of the present invention. However, it will be recognizedby one of ordinary skill in the art that the present invention may bepracticed without these specific details. In other instances, well-knownmethods, procedures, components, and circuits have not been described indetail as not to unnecessarily obscure aspects of the embodiments of thepresent invention.

Embodiments of the present invention provide a method and system forusing external storage to amortize CPU cycle utilization. Embodiments ofthe present invention implement a computer software translation processwhile simultaneously reducing the overhead penalty imposed on overallcomputer system performance. Embodiments of the present invention useexternal storage to amortize CPU cycle utilization, wherein translatedinstructions are stored in a storage medium and subsequently accessed ona subsequent execution of a non-native application in order to amortizeCPU cycles used in generating the translated instructions. Embodimentsof the present invention and their benefits are further described below.

Notation and Nomenclature

Some portions of the detailed descriptions which follow are presented interms of procedures, steps, logic blocks, processing, and other symbolicrepresentations of operations on data bits within a computer memory.These descriptions and representations are the means used by thoseskilled in the data processing arts to most effectively convey thesubstance of their work to others skilled in the art. A procedure,computer executed step, logic block, process, etc., is here, andgenerally, conceived to be a self-consistent sequence of steps orinstructions leading to a desired result. The steps are those requiringphysical manipulations of physical quantities. Usually, though notnecessarily, these quantities take the form of electrical or magneticsignals capable of being stored, transferred, combined, compared, andotherwise manipulated in a computer system. It has proven convenient attimes, principally for reasons of common usage, to refer to thesesignals as bits, values, elements, symbols, characters, terms, numbers,or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the followingdiscussions, it is appreciated that throughout the present invention,discussions utilizing terms such as “storing” or “accessing” or“exporting” or “retrieving” or “translating” or the like, refer to theaction and processes of a computer system (e.g., computer system 500 ofFIG. 5), or similar electronic computing device, that manipulates andtransforms data represented as physical (electronic) quantities withinthe computer system's registers and memories into other data similarlyrepresented as physical quantities within the computer system memoriesor registers or other such information storage, transmission or displaydevices.

Embodiments of the Present Invention

FIG. 1 shows a diagram of the operation of a system 100 used toimplement an instruction translation process in accordance with oneembodiment of the present invention. As depicted in FIG. 1, system 100includes a non-native application 101 having its constituentinstructions fetched from its storage location (e.g., memory, disk,network, etc.) to build a sequence 110, or block, of non-nativeinstructions for processing by an instruction translator 120. Theinstruction translator 120 translates the non-native instructions111-114 to generate a corresponding sequence 130 (e.g., block) of nativeinstructions. The native instructions 131-134 comprising the sequence130 are then executed by the CPU hardware 140 of the system 100.

Thus, the system 100 embodiment shown in FIG. 1 implements a translationprocess wherein the non-native application 101 is executed by the system100 and, for example, the functionality of the non-native application101 is provided to a user.

Depending upon the specific requirements of a particular implementation,the non-native application 101 can be an x86 application and thenon-native instructions 111-114 comprising the application 101 can bex86 instructions. The CPU hardware 140 executing the translated nativeinstructions can be a VLIW (very long instruction word) CPU hardwareunit and the native instructions 131-134 fed into the CPU hardware 140can be VLIW instructions. In such an implementation, the VLIWinstructions would be configured (e.g., by the instruction translator120) to efficiently feed multiple pipeline front ends of the CPUhardware 140 to achieve maximum concurrency and parallelism.Descriptions of processor architectures implementing translation can befound in commonly assigned U.S. Pat. No. 5,958,061, HOST MICROPROCESSORWITH APPARATUS FOR TEMPORARILY HOLDING TARGET PROCESSOR STATE, which isincorporated herein in its entirety.

It should be noted that other types of non-native applications can beused in the translation process of the present invention, in addition tox86 applications (e.g., SPARC, MIPS, etc.). Similarly, the nativeinstructions generated by the translation process can be other types ofinstructions besides VLIW instructions. Thus, at times it can be helpfulfor the reader's understanding to use the terms “target instruction” and“target application” to refer to the non-native instructions and thenon-native application, while the terms “host instruction” and “hostarchitecture” are used to refer to the native instructions and the CPUhardware that executes the native instructions.

FIG. 2 shows a diagram of the basic components of a system 200 used toimplement an instruction translation process in accordance with oneembodiment of the present invention. As depicted in FIG. 2, system 200shows a processor environment 201 including an instruction translator220, a cache 225 storing the translated instructions 230, and CPUhardware 240.

The system 200 embodiment functions by implementing a method for usingexternal storage to amortize CPU cycle utilization while executing theinstruction translation process. In the present embodiment, non-nativeinstructions from the non-native application (e.g., application 101 ofFIG. 1) are fetched and translated by the instruction translator 220.The instruction sequence from the non-native application is translatedby the instruction translator 220 into resulting translated instructionswhich can natively execute on the CPU hardware 240. The resultingtranslated instructions 230 are stored within the cache memory 225. Thetranslated instructions 230 are executed by the CPU hardware 240 toimplement the functionality of the non-native application.

Thus, in order to execute the instructions of the non-nativeapplication, the computer readable code comprising the non-nativeapplication is translated first into the translated instructions 230which then execute very efficiently on the CPU hardware 240. Dependingupon the specifics of the non-native application, the non-nativeapplication is entirely translated, or only portions (e.g., thoseportions of the non-native application which are most CPU intensive) ofthe non-native application are translated.

In accordance with embodiments of the present invention, the translatedinstructions 230, once generated by the instruction translator 220, arestored within the cache 225 and reused as often as possible. By reusingthe translated instructions 230, the system 200 avoids having to executethe instruction translator 220 multiple times to translate the samenon-native instructions. For example, as long as the non-nativeapplication executes in-context (e.g., as a process executing on theprocessor environment 201 as opposed to being context switched out to aswap file), the translated instructions 230 can be executed directly onthe CPU hardware 240 without having to constantly re-translate theinstructions using the instruction translator 220. This greatly reducesCPU overhead involved in executing the overall translation process. Thereusing of the translated instructions 230 to implement the applicationfunctionality essentially takes the instruction translator 220 “out ofthe loop” and allows the CPU cycles consumed by the instructiontranslation process to be saved, along with saving the power consumed byexecuting the instruction translation process.

In accordance with embodiments of the present invention, in order toleverage the CPU cycle savings and the power savings provided by thereuse of the translated instructions 230, additional external memory ofthe storage medium 250 is used to save larger amounts of translatedinstructions 241. As shown in FIG. 2, the translated instructions 230are stored within a cache 225. To increase the amount of memoryavailable for storage, translated instructions can be saved outside ofthe cache 225 to the external storage medium 250 for later reuse. Thisgreatly increases the storage area available for storing translatedinstructions. In so doing, the translated instructions resulting fromvery large non-native applications, or many different non-nativeapplications, can be stored and later accessed to in order to amortizethe CPU cycles (e.g., CPU cycles required to implement the translationprocess, power consumed by the translation process, and the like)required to generate the translated code.

Thus, when a non-native application which has been previously translatedby the instruction translator 220 is subsequently encountered (e.g.,after a context switch), the translated instructions (e.g., thetranslated instructions 241) corresponding to the application can beaccessed from the storage medium 250, or retrieved from the storagemedium 250 into the cache 225, and directly executed on the CPU hardware240.

The storing of the translated instructions 241 provides additionalbenefits when the storage medium 250 is non-volatile. For example, insuch case, when a non-native application is first started up, thenon-native application can be instantiated completely from, or primarilyfrom, the stored translated instructions 241 as opposed to thenon-native application being read from the non-volatile storage medium,first translated by the instruction translator 220, and theninstantiated. This greatly reduces the amount of time (e.g., CPU cycles)required to first start up the non-native application. As long aspreviously translated instructions 241 reside within the storage medium250 (e.g., from a previous computing session), on a subsequent executionof the application, the operation of the instruction translator 220 canbe avoided, or minimized, and the CPU cycles and power required tooperate the instruction translator 220 can be amortized.

It should be noted that in one embodiment, the instruction translator220 can be implemented as a hardware based instruction decoder 220 asfound in many types of microprocessor architectures. In such embodiment,the instruction decoder 220 takes instructions for the application(e.g., x86 instructions) and decodes those instructions into “pipelineinstructions” or “micro-architecture instructions” that efficiently feedthe CPU hardware 240. For example, many processor architectures (e.g.,processor environments) decode x86 instructions into more simpleRISC-like micro-architecture instructions which then execute on theinternal CPU hardware of the processor architecture. Thesemicro-architecture instructions are then scheduled by the instructiondecoder 220 such that they fill the multiple pipelines to achievemaximum usage and minimal stalling. In such architectures, theinstruction decoder generally operates continuously, continuouslydecoding the x86 instructions into the constituent micro-architectureinstructions which are then executed by the CPU hardware. In oneembodiment, the decoded/translated instructions include higher-leveloptimizations such as, for example, speculation, loop unrolling, oroperation reordering, in order to efficiently feed the CPU hardware.

In accordance with embodiments of the present invention, theseconstituent micro-architecture instructions comprise the translatedinstructions (e.g., translated instructions 230) which can be saved forreuse in the manner described above. By saving the translatedinstructions (translated instructions 241), embodiments of the presentinvention can implement the saving of the micro-architecture state ofthe processor environment 201. For example, in addition to savingmicro-architecture instructions corresponding to arithmetic operations(integer additions, floating point, etc.), and the efficient order toexecute them in, micro-architecture instructions corresponding to branchprediction information can also be saved. Thus, in such an embodiment,the target instructions are the x86 instructions and the hostinstructions are the micro-architecture instructions which execute onthe CPU hardware 240. The instruction translator/instruction decoder 220can be software based, hardware based, or some combination thereof.

FIG. 3 shows a diagram of the basic components of a system 300 used toimplement an instruction translation process in accordance with oneembodiment of the present invention. As depicted in FIG. 3, system 300shows a CPU integrated circuit 301 including an instruction translator320, a cache 325 storing the translated instructions 330, and CPUhardware 340.

In the system 300 embodiment, the CPU integrated circuit 301 providesthe processor environment for the instruction translator 320, the cache325 storing the translated instructions 330, and the CPU hardware 340.The system 300 embodiment also shows the saving of translatedinstructions 341 into system memory 350 and the saving of translatedinstructions 342 into disk storage 360. In other respects, the system300 embodiment functions in a manner substantially similar to the system200 embodiment described above in the discussion of FIG. 2. Target, ornon-native, applications 355 and/or applications 356 are instantiated byfirst translating their instructions using the instruction translator320 and storing the resulting translated instructions into the cachememory 325. Thus, on subsequent executions of the applications 355and/or applications 365, the CPU cycles consumed by the operation of theinstruction translator 320 are then amortized by accessing storedtranslated instructions 341 or stored translated instructions 342 andexecuting the stored translated instructions directly on the CPUhardware 340. The objective is to opportunistically access the storedinstructions 341-342 (e.g., through a number of different computerimplemented means such as DMA transfer, etc.) and execute them on theCPU hardware 340.

The system 300 embodiment explicitly shows storing the translatedinstructions 330 in both the system memory 350 and the disk storage 360.Although the system memory 350 is generally volatile, the system memory350 typically has a much larger storage capacity in comparison to thecache memory 325. The disk storage 360 provides even greater storagecapacity in comparison to the system memory 350. Additionally, the diskstorage 360 is non-volatile, and can thus store the translatedinstructions 342 for use during multiple computing sessions (e.g.,wherein the computer system is turned off between sessions). However,access to the translated instructions 342 is generally slower thanaccess to the translated instructions 341 residing in system memory 350.

FIG. 4 shows a flowchart of the steps of a process 400 in accordancewith one embodiment of the present invention. As depicted in FIG. 4,process 400 shows the steps involved in a translation CPU cycleamortization process as implemented by a computer system (e.g., system300) in accordance with one embodiment of the present invention.

Process 400 begins in step 401, where an initial execution of anon-native, or target, application is begun. In step 402, translatedinstructions are generated using a instruction translator (e.g.,instruction translator 320) within a processor environment (e.g., CPU301). In step 403, the resulting translated instructions are storedwithin a cache memory within the processor environment. In step 404, asthe translated instructions are being stored within the cache, thetranslated instructions are executed by the CPU hardware. In step 405,the translated instructions are stored in a storage medium (e.g., systemmemory 350) external to the processor environment. As described above,the translated instructions can be stored within, for example, systemmemory, disk storage, or other types of storage media external to theprocessor environment. In step 406, upon a subsequent execution of thenon-native application, the translated instructions stored in thestorage medium are accessed and are directly executed by the CPUhardware of the processor.

Thus, embodiments of the present invention provide a method and systemfor using external storage to amortize CPU cycle utilization.Embodiments of the present invention implement a computer softwaretranslation process while simultaneously reducing the overhead penaltyimposed on overall computer system performance. Embodiments of thepresent invention use external storage to amortize CPU cycleutilization, wherein translated instructions are stored in a storagemedium and subsequently accessed on a subsequent execution of anon-native application in order to amortize CPU cycles used ingenerating the translated instructions.

Computer System Platform

With reference now to FIG. 5, a computer system 500 in accordance withone embodiment of the present invention is shown. Computer system 500shows the general components of a computer system in accordance with oneembodiment of the present invention that provides the execution platformfor implementing certain software-based functionality of the presentinvention. As described above, certain processes and steps of thepresent invention are realized, in one embodiment, as a series ofinstructions (e.g., software program) that reside within computerreadable memory units of a computer system (e.g., system 500) and areexecuted by the CPU 501 of system 500. When executed, the instructionscause the system 500 to implement the functionality of the presentinvention as described above.

In general, system 500 comprises at least one CPU 501 coupled to a Northbridge 502 and a South bridge 503. The North bridge 502 provides accessto system memory 515 and a graphics unit 510 that drives a display 511.The South bridge 503 provides access to a plurality of coupledperipheral devices 531-533 as shown. Computer system 500 also shows aBIOS ROM 540 that stores BIOS initialization software.

The foregoing descriptions of specific embodiments of the presentinvention have been presented for purposes of illustration anddescription. They are not intended to be exhaustive or to limit theinvention to the precise forms disclosed, and obviously manymodifications and variations are possible in light of the aboveteaching. The embodiments were chosen and described in order to bestexplain the principles of the invention and its practical application,to thereby enable others skilled in the art to best utilize theinvention and various embodiments with various modifications as aresuited to the particular use contemplated. It is intended that the scopeof the invention be defined by the claims appended hereto and theirequivalents.

1. A method comprising: storing a plurality of translated instructionsin a location external to a processor, wherein the translatedinstructions correspond to a non-native application; if execution of thenon-native application is requested after a context switch, accessingthe translated instructions from the location external to the processor;and after said accessing, executing in the processor the translatedinstructions.
 2. The method of claim 1, wherein said storing comprises:storing the translated instructions in a cache memory external to theprocessor.
 3. The method of claim 1, wherein said storing comprises:storing the translated instructions in a system memory external to theprocessor.
 4. The method of claim 1, wherein said storing comprises:storing the translated instructions in a disk memory external to theprocessor.
 5. The method of claim 1, wherein the translated instructionsinclude pipeline instructions for the processor.
 6. The method of claim5, wherein the pipeline instructions comprise very long instruction word(VLIW) instructions.
 7. The method of claim 1, wherein the translatedinstructions include a micro-architecture state of the processor.
 8. Themethod of claim 1, wherein the translated instructions include branchprediction information.
 9. The method of claim 1 further comprising:generating the translated instructions by using an instruction decoder.10. An apparatus comprising: means for executing a plurality oftranslated instructions, wherein the translated instructions correspondto a non-native application; means for storing the translatedinstructions externally relative to the means for executing; and meansfor accessing the translated instructions from the means for storing ifexecution of the non-native application is requested after a contextswitch.
 11. The apparatus of claim 10, wherein the means for storingcomprises: a cache memory external to the means for executing.
 12. Theapparatus of claim 10, wherein the means for storing comprises: a systemmemory external to the means for executing.
 13. The apparatus of claim10, wherein the means for storing comprises: a disk memory external tothe means for executing.
 14. The apparatus of claim 10, wherein thetranslated instructions include pipeline instructions for the means forexecuting.
 15. The apparatus of claim 14, wherein the pipelineinstructions comprise very long instruction word (VLIW) instructions.16. The apparatus of claim 10, wherein the translated instructionsinclude a micro-architecture state of the means for executing.
 17. Theapparatus of claim 10, wherein the translated instructions includebranch prediction information.
 18. The apparatus of claim 10 furthercomprising: means for generating the translated instructions.
 19. Acomputer-readable medium comprising computer-executable instructionsstored therein, said computer-executable instructions comprising:instructions to store a plurality of translated instructions in alocation external to a processor, wherein the translated instructionscorrespond to a non-native application; instructions to access thetranslated instructions from the location external to the processor ifexecution of the non-native application is requested after a contextswitch; and instructions to execute in the processor the translatedinstructions after performing said instructions to access.
 20. Thecomputer-readable medium of claim 19, wherein said instructions to storecomprise: instructions to store the translated instructions in a cachememory external to the processor.
 21. The computer-readable medium ofclaim 19, wherein said instructions to store comprise: instructions tostore the translated instructions in a system memory external to theprocessor.
 22. The computer-readable medium of claim 19, wherein saidinstructions to store comprise: instructions to store the translatedinstructions in a disk memory external to the processor.
 23. Thecomputer-readable medium of claim 19, wherein the translatedinstructions include pipeline instructions for the processor.
 24. Thecomputer-readable medium of claim 23, wherein the pipeline instructionscomprise very long instruction word (VLIW) instructions.
 25. Thecomputer-readable medium of claim 19, wherein the translatedinstructions include a micro-architecture state of the processor. 26.The computer-readable medium of claim 19, wherein the translatedinstructions include branch prediction information.
 27. Thecomputer-readable medium of claim 19, wherein the computer-executableinstructions further comprise: instructions to generate the translatedinstructions by using an instruction decoder.